The present invention relates to computer-based systems for enhancing collaboration between and among individuals who are separated by distance and/or time. Remote presentation is required for this distance collaboration. Ideally, the full range, level and intensity of interpersonal communication and information sharing will be provided with such remote presentation.
Screen capture and processing capabilities have recently been integrated into desktop and portable personal computers and workstations. While such systems are capable of processing, combining, and recording video and data locally networked collaborative environments are not adequately supported, principally due to the substantial bandwidth requirements and high latency for real-time transmission of high-quality, digitized audio and full-motion. Therefore, a number of sampling techniques are typically used when sending remote-presentation screen.
There are two main color spaces from which the majority of video formats are derived. The first color space is commonly referred to as the RGB (Red Green Blue) color space (hereinafter referred to as RGB). RGB is used in computer monitors, cameras, scanners, and the like. The RGB color space has a number of formats associated with it. Each format includes a value representative of the Red, Green, and Blue chrominance for each pixel. In one format, each value is an eight bit byte. Therefore, each pixel consumes 24 bits (8 bits (R)+8 bits (G)+8 bits (B)). In another format, each value is 10 bits. Therefore, each pixel consumes 30 bits.
Another color space widely used in television systems and is commonly referred to as the YCbCr color space or YUV color space (hereinafter referred to as YUV). In many respects, YUV provides superior video quality in comparison with RGB at a given bandwidth because YUV takes into consideration that the human eye is more sensitive to variations in the intensity of a pixel than in its color variation. As a result, the color difference signal can be sub-sampled to achieve bandwidth saving. Thus, the video formats associated with the YUV color space, each have a luminance value (Y) for each pixel and may share a color value (represented by U and V) between two or more pixels. The value of U (Cb) represents the blue chrominance difference between B-Y and the value of V (Cr) represents the red chrominance difference between R-Y. A value for the green chrominance may be derived from the Y, U, and V values. YUV color space has been used overwhelmingly in video coding field.
For convenience and keeping with conventional video techniques, the following discussion describes each block as representing one pixel. Therefore, hereinafter, the term pixel will be used interchangeably with the term block when referring to arrays depicted in any illustrations.
There are several YUV formats currently existing.
In the YUV444 format, each pixel is represented by a Y, U, and V value. The YUV444 format uses eight bits for the Y value, eight bits for the U value, and eight bits for the V value. Thus, each pixel is represented by twenty-four bits. Because this format consumes twenty-four bits for each pixel, other YUV formats are down-sampled from the YUV444 format so that the number of bits per pixel is reduced. The reduction in bits per pixel provides improvement in streaming efficiency. However, down-sampling results in a corresponding degradation in video quality.
For the YUV420 format only one pixel per 2×2 array of pixels is represented by twenty-four bits. The other pixels in 2×2 array are each represented by eight bits of Y value only. For example, using matrix notation, (1,1) would be represented by 8 bits each of the Y, U and V components while (1,2), (2,1) and (2,2) would each be represented only by 8 bits of Y component. Thus average number of bits per pixel in the YUV420 format is twelve bits. The YUV420 is a planar rather than packed format. Thus, the YUV420 data is stored in memory such that all of the Y data is stored first, then the U data, then all of the V data.
Based on the quality that is desired and the transmission bandwidths that are available, an electronic device manufacturer may design their electronic devices to operate with either of the YUV444 or YUV420 formats. However, when transmission bandwidths increase and/or consumers begin to demand higher quality video, the existing electronic devices will not support the higher quality video format. For example, currently many digital televisions, set-top boxes, and other devices are designed to operate with the YUV420 video format. In order to please the different categories of consumers, there is a need to accommodate both video formats.
The video codecs and picture codecs are being used to encode and decode the screen data for remote presentation sessions. The remote presentation sessions typically require high quality that can only be achieved by coding using YUV444 format without sub-sampling to other formats such as YUV420 or YUV422. The video codecs have some drawbacks such as high encoding latency and decoding supported typically limited to YUV420 formats. Though the picture codecs such as JPEG and JPEG2000 support low encoding latency and YUV444, they typically compress less as compared to video codecs. This limits them to local area networks as they cannot support low bandwidth requirements of wide area networks. Also, the current codecs used for the remote presentation session do not incorporate scaling techniques as applies to quality, temporal and spatial scalability to improve the overall system performance.
Because of bandwidth constraint of the wide area networks and low latency requirements of the remote display sessions, existing systems use compression systems that are less efficient. The existing systems use less efficient compression techniques as video codecs reduce the quality to meet with bandwidth constraints of wide area networks and increase the latency. Both conditions critically effect remote display sessions.
Due to growing demands of more efficient codecs, it is apparent that new techniques for remote presentation sessions are required to support YUV444 format with high compression and support for various scalability options. Therefore, for all the above reasons, developing a new technique for efficiently encoding and decoding is important for the remote presentation session applications.